Agenda

• A. The Routing Problem — Why tool selection matters (~10 min)
• B. Classifier-Based Routing — LLM categorizes queries (~15 min)
• C. Semantic Routing — Embeddings for tool matching (~20 min)
• D. Programmatic Tool Calling — Batch tools without round-trips (~15 min)
• E. Wrap-up — Key takeaways & lab preview (~5 min)

The Tool Proliferation Problem

Your agent starts with 3 tools. Then it grows:

• search_web, search_news, search_academic
• get_stock_price, get_financial_report, calculate_roi
• read_pdf, summarize_document, extract_tables
• send_email, create_calendar_event, query_database

Why More Tools ≠ Better Agent

The Solution: Dynamic Tool Routing

graph LR
    Q["User Query"] --> R["Router"]
    R -->|"Financial"| TF["Finance Tools<br/>(3 tools)"]
    R -->|"Academic"| TA["Academic Tools<br/>(4 tools)"]
    R -->|"General"| TG["General Tools<br/>(5 tools)"]
    TF --> A["Agent"]
    TA --> A
    TG --> A

    style R fill:#1C355E,stroke:#00C9A7,color:white
    style Q fill:#00C9A7,stroke:#1C355E,color:#1C355E
    style A fill:#FF7A5C,stroke:#1C355E,color:#1C355E

The router selects a subset of tools before the agent runs.

How Classifier Routing Works

sequenceDiagram
    participant U as User
    participant C as Classifier LLM
    participant T as Tool Registry
    participant A as Agent

    U->>C: "What's Apple's stock price?"
    C->>C: Classify domain
    C-->>C: Domain = "Financial"
    C->>T: Get tools for "Financial"
    T-->>C: [get_stock_price, calculate_roi]
    C->>A: Agent + 2 tools
    A->>U: "$178.72"

The Classifier Prompt

A small, fast model (GPT-4o-mini, Claude Haiku) classifies the query:

CLASSIFIER_PROMPT = """Classify this query into ONE domain.

Domains: financial, academic, general, technical

Query: {query}

Respond with ONLY the domain name, nothing else.
"""

When Classifier Routing Works Well

From Categories to Similarity

Problem with classifiers: You must define categories upfront.

Semantic routing: Match the query to tools based on meaning, not labels.

graph LR
    Q["Query Embedding"] --> S1["Tool 1<br/>similarity: 0.92"]
    Q --> S2["Tool 2<br/>similarity: 0.87"]
    Q --> S3["Tool 3<br/>similarity: 0.45"]
    Q --> S4["Tool 4<br/>similarity: 0.23"]
    
    S1 --> T["Top-K Selected"]
    S2 --> T

    style Q fill:#1C355E,stroke:#00C9A7,color:white
    style S1 fill:#00C9A7,stroke:#1C355E,color:#1C355E
    style S2 fill:#00C9A7,stroke:#1C355E,color:#1C355E
    style T fill:#FF7A5C,stroke:#1C355E,color:#1C355E

How Semantic Routing Works

1. Index phase: Embed all tool descriptions once at startup
2. Query phase: Embed the user’s query
3. Match phase: Compute cosine similarity, select top-K tools

# Index phase (once)
tool_embeddings = {
    "search": embed("search: Find information on the web"),
    "calculate": embed("calculate: Evaluate mathematical expressions"),
    "get_stock": embed("get_stock_price: Get current stock price"),
}

# Query phase (per request)
query_embedding = embed("What is Apple's stock worth?")
similarities = cosine_similarity(query_embedding, tool_embeddings)
top_tools = select_top_k(similarities, k=3)

Cosine Similarity Explained

Measures the angle between two vectors:

\[\text{similarity} = \frac{A \cdot B}{\|A\| \|B\|}\]

The RoutedAgent Pattern

Combine routing with the agent loop for efficient execution:

class RoutedAgent:
    def run(self, query: str) -> str:
        selected_tools = self.router.select_tools(query)   # Route first
        return self.base_agent.run(query, tools=selected_tools)  # Then act

Context Window Savings

Classifier vs Semantic Routing

When to Use Routed Agents

The Round-Trip Problem

Traditional tool calling: N tools = N model round-trips

sequenceDiagram
    participant M as Model
    participant T as Tools
    
    M->>T: Call tool_1
    T-->>M: Result (in context)
    M->>T: Call tool_2
    Note over M,T: ...
    M->>T: Call tool_n
    T-->>M: Result (in context)
    M->>M: Final answer

The allowed_callers Pattern

Mark tools as callable from code execution:

tools = [
    {"type": "code_execution_20260120", "name": "code_execution"},
    {
        "name": "query_database",
        "description": "Execute a SQL query. Returns JSON.",
        "input_schema": {"type": "object", "properties": {"sql": {"type": "string"}}},
        "allowed_callers": ["code_execution_20260120"],  # <-- Key field
    },
]

Token & Latency Gains

Advanced Patterns

Early termination — stop as soon as success criteria met:

for endpoint in ["us-east", "eu-west", "apac"]:
    status = await check_health(endpoint)
    if status == "healthy":
        print(f"Found healthy: {endpoint}")
        break

Conditional tool selection — choose tool based on data:

file_info = await get_file_info(path)
if file_info["size"] < 10000:
    content = await read_full_file(path)
else:
    content = await read_file_summary(path)

When to Use Programmatic Calling

Alternative: Self-Managed Execution

Not using Claude? You can implement the pattern yourself:

1. Give Claude a code execution tool (e.g., Python sandbox)
2. Describe available functions in that environment
3. Claude writes code → you execute → return result

# Your code execution tool
{
    "name": "execute_python",
    "description": "Run Python code. Available functions: search(), query_db(), send_email()",
    ...
}

Trade-off: More control, but you manage security and infrastructure.

Key Takeaways

1. Tool proliferation hurts performance — route to reduce context bloat
2. Classifier routing uses a small LLM to categorize queries (fast, rigid)
3. Semantic routing uses embeddings to match queries to tools (flexible, scalable)
4. Programmatic calling eliminates round-trips — Claude writes code that calls tools
5. Combine patterns: route to select tools, programmatic calling to execute efficiently

Lab Preview: Efficient Tool Calling

Questions?